Evaluation of Large-scale Storage Systems for Satellite Data in Geo Grid

The Global Earth Observation (GEO) Grid infrastructure is an E-Science infrastructure which enables global research activity and drives geosciences to get a significant discovery or achievement in their fields. The main design principal of the GEO Grid system is the open and standard protocol-based architecture. It supports creation of a virtual organization (VO) by integrating data and computing services according to the requirements by the VO. VO-level access control realizes flexible and scalable security infrastructure, against the increasing number of VOs and users. For extending the GEO Grid infrastructure, the number of resource providers should be more. Though the GEO Grid framework provides a facilitating tool of managing a GEO Grid site, building and operating a large storage system is one of the difficult tasks. For example, the ASTER storage which joins the GEO Grid system is now operated by AIST and it is faced on the capacity issue for new sensors. In order to build a larger, petabytes-scale storage system, a Gfarm-based storage system and a Lustre-based storage system, were compared. Real data sets of the satellite images were imported into both storage systems and the performance was measured by a practical application. Functions for fault tolerance and daily maintenance work issues are investigated to reveal operation cost. The comparison results indicate that a factor of choosing storage system is not performance but installation and operation cost. This paper provides an overview of the GEO Grid system, and summary information of what the GEO Grid resource providers should consider about for their internal storage systems.